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ABSTRACT 

This research project examines the relationship between modality of course delivery 
and the level of student GPA, satisfaction and retention achieved for students 
attending either traditional On-site or Hybrid (partial Online and partial On-site) 
delivery modality university courses. The research project incorporates data from 110 
courses and 164 students. The results, indicate that there is not a statistically 
significant difference, between the levels of student satisfaction, student retention or 
grade point average between students taking classes in the traditional on-site 
modality vs. those attending class via the hybrid modality. 
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INTRODUCTION 

Online education has been gaining momentum in today's educational institutions. 
With recent technological advances in the worldwide web, a student is no longer 
required to reside in the same place as his or her college. These technological 
advances have sparked an increasing number of educational institutions to offer 
distant learning programs in both the U.S. and internationally. According to the 
National Center of Educational Statistics, 44% of higher education institutions in the 
U.S. now offer distant learning programs (Bartley & Golek, 2004). 

Why has distance learning exploded so rapidly in the last decade across the U.S.? 
Increasing trends are showing that students are no longer allowing cultural, 
geographic, and socioeconomic boundaries to prevent them from pursuing higher 
education. Although online education is growing, many questions are surfacing 
around the effectiveness of this type of education. Arbaugh (2000) supports this 
position making the point that our knowledge of what makes online courses effective 
learning experiences is limited. 

Purpose 

There are a number of commonly employed measures of program success in 
education. Among these, student satisfaction, student retention and student grade 
point average (GPA) are among the more frequently employed. The purpose of this 
study was to determine the effect of the modality of course delivery, whether 
traditional on-site or hybrid, on student satisfaction, student retention and student 
grade point average, lliese three factors are among the major issues concerning 
higher education today as the environment becomes increasingly competitive. Central 
among these issues is student retention. 

Modalities: 
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> On-site: the traditional face-to-face classroom environment with student 
and faculty in the same room. Students meet on-site each week and 
engage in interactive instruction as well as possibly meeting weekly in 
smaller learning teams outside the university classroom to work on group 
projects. 

> Hybrid: a combination of on traditional on-site campus and online 

instruction that is structured for students that require flexible schedules. A 
student who enrolls in a hybrid course attends the first 4 hours and last 4 
hours of class in traditional face-to-face class sessions. The rest of the 
classroom sessions are held entirely online. It should be noted that other 
than the desire for a more flexible schedule the overall demographics of 
students taking hybrid courses is the same as those of students attending 
on-site classes at the university being studied. Prior studies at the 
university have established that there is no demographic difference 
between students electing to enroll in the different modalities in terms of 
average age, educational history, or family status . (Carmel, 2006) 

REVI EW OF LITERATURE 

Student Satisfaction 

In regard to student satisfaction, choice seems to play an important role in how 
individual students determine if they are satisfied. As adult learners, it is important 
for the university to treat its' students with respect and to acknowledge the 
accomplishments achieved in their current careers. Yatrakis (2002) makes the point 
that the more time students spend interacting with classmates, the higher the 
satisfaction level as well as the retention level of information. Yatrakis (2002) also 
observed that student dissatisfaction does not automatically mean drop-off or 
withdrawal from the program. 

Haythomthwaite (2000), found that an on-site "boot camp" preceding online courses 
can help build a sense of community among distance learning students and enhance 
their satisfaction and learning outcomes. This is supported by Gold (2004) who found 
that increased levels of software mediated interaction in online classes resulted in 
higher final exam scores. The hybrid course delivery modality builds upon these 
findings by having the students meet first in a traditional on-site setting prior to 
moving to the online environment. 

Arguably the end results of online education may be similar or even better than 
traditional on-site formats. This has been well established by research under the 
general category of "No Significant Difference", with the initial study published by 
Russell (2001). One study attempted to remove professor- bias by blind-scoring tests 
in a graduate- level online vs. traditional course environment. The results indicated, 
"...average score for the online class was 5 points (5% ) higher than for the on 
campus class. "(Fallah & Ubell, 2000). This is further supported by Gold (2004) who 
states that there is overwhelming evidence that instruction delivered using online 
technology is equivalent to conventional instruction when using student achievement 
as the outcome measure. 

Retention 

Retention is important for both the students and the University. The students can 
complete the degree that they were striving for and the University is able to complete 
the goal of retaining the student. However, "anecdotal evidence and studies by 
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individual institutions suggest that course-completion and program retention rates 
are generally lower for distance-education courses than in their on-site counterparts" 
(Carr, 2000, p. A39). 

There is an abundance of literature on student retention for online courses, but as 
Yatrakis and Simon (2002) state, its main theme is comparisons between completely 
online and completely on-site modalities. Research related to the effect of the 
completely on-site m^ality vs. the hybrid modality combining on-site and online 
studies is limited because of the relatively new implementation of hybrid courses. 

According to eLearn Magazine (n.d.), "keeping students enrolled in online courses can 
be a struggle. Some colleges offer proof that online retention lags behind brick-and- 
mortar retention. Washington Online - Washington State's online division for 
community college - claims a retention rate of 70% for online students versus 85% 
for the state's on-site community college students. Many researchers have found 
significant differences in retention between the two modalities of online and on-site. 
Some, have cited student concerns about instructional quality in online courses 
(Bloom 1998; Terry 2000), while others consider virtual courses an "inferior 
technology," particularly in the teaching of complex material (Farrington, 1999; 
Brown and Liedholm, 2002). 

In contrast, the University of Central Florida (UCF) provides data showing that 
student retention in hybrid courses is better than retention in completely online 
courses and equivalent to that of completely on-site courses. (Dziuban, C. D. et al, 
2001). According to Robertson (2003) the College of the Mainland proposal states 
that hybrid classes have the potential to improve retention in both online and on-site 
courses. 

Grade Point Average (GPA) 

Students who reported higher levels of satisfaction tended to have higher grades and 
were more likely to have completed their program than students who were less 
satisfied. That is, student satisfaction is positively associated with program 
completion rates and overall GPA. These findings are similar regardless of gender, 
age, program, or location of institution (BC College, 2002). 

Methodology 

Test Design 

The study used a non-equivalent Group Design (NEGD) also known as a quasi-test 
design. "A quasi-experimental design is one that looks a bit like an experimental 
design but lacks the key ingredient — random assignment... I n the NEGD, we most 
often use intact groups that we think are similar as the treatment and control groups. 
I n education, we might pick two comparable classrooms or schools" (Trochim, 2006). 
While an attempt is made to assure that the two groups are as similar as possible it is 
not possible for the researcher to control the assignment to the groups on a random 
basis. This makes the NEGD inherently subject to internal validity threats which need 
to be addressed. 

The primary such threat is the threat of selection on internal validity creating a 
selection bias in the study. This bias is the risk that any factor other than the ones 
being analyzed may have lead to the result observed. There are a number of selection 
bias threats in a multiple group study the most relevant are Instrumentation and 
Selection History. 
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The key to addressing these validity issues is to assure that the groups are as 
equivalent as they can be made given the nature of the environment and that the 
methodology is applied in a consistent manner. The validity issues were mitigated as 
follows in this study. Random selection of survey participants (students) was made in 
each group so that there was no bias as to the prior history or accomplishments of 
the students. All students were students were dedicated members of a given group or 
course modality for the duration of their program so there is no issue related to a 
student being a member of both sample groups. Sampling a large number of 
students, spread across a wide variety of classes in each modality and using a single 
survey instrument for all students also mitigated the internal validity issues. 

Survey 

The sample in this study consisted of undergraduate and graduate students enrolled 
at two campuses of a regionally accredited university. The analysis studied the 
differences, if any, between hybrid course students and traditional on-site course 
students regarding their grade point average (GPA), satisfaction, and retention. I n all 
cases, students were dedicated to a given modality for the duration of their program. 

The sample data was collected by asking students to fill out a survey questionnaire. 
The respondents were informed of the purpose of the research, which was self- 
administered. I n addition, they were informed that after filling out the questionnaire, 
they would be asked to not discuss their responses with other participants to avoid 
the risk of biased information. All non-specific survey questions were written so that 
the answers would fit appropriately into a standard (1-5 point scale) Likert Scale for 
analysis. The survey instrument was designed with a wide range of questions to allow 
for a variety of analyses. The questions listed below are those which were relevant to 
the study presented in this paper. 

Satisfaction 

The following questions were asked in the satisfaction portion of the administered 
survey questionnaire: 

> Are the class sizes adequate? 

> Are student services staff helpful and courteous? 

> Are your professors knowledgeable about the class subjects? 

> I s the university Website easily navigated? 

> How would you rate your overall educational experience at this 
university? 

Retention 

Retention level was measured to answer the question of whether there was a 
statistically significant difference in retention between hybrid and on-site students. 
The following questions in the survey questionnaire were designed to address the 
retention issue: 

> Would you consider continuing your education at this university? 

> Approximately how long have you been a student at this university? 

> How many classes have you taken at this university? 

> Do you feel that the quality of education at this university is high? 

> Would you recommend that other students pursue their education at 
this university? 

> Do you consider the university to be helpful in networking with peers 
that will assist in your future professional endeavors? 
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GPA 

The following questions were asked in the GPA portion of the administered survey 
questionnaire: 

What is your overall GPA at this time? 

It should be noted that the possiblity exists that some students did not provide a 
correct answer to this question. For purposes of this study it was assumed that the 
students were in fact providing accurate information, especially since they were 
assured of confidentiality and anonymity by the research team. In any event, the 
large sample size involved in calculating the mean value would tend to mitigate this 
issue. 

Participants 

I n order to study these factors, responses from students were collected and tabulated 
from a sample of 164 students in 110 courses who chose to enroll in courses that 
were available either on-site or hybrid formats. The sample consisted of 95 female 
students (58% ) and 69 male students (42% ) with the following ethnicity 
breakdown: 65 African American students (40% ), five Asian students (3% ), 30 
Caucasian/ White students (18%), 61 Hispanic/ Latino students (37%) and three 
Native Hawaiian/ Pacific Islander students (2%). Out of 164 students, 95 were 
attending on-site classes and 69 students were attending hybrid courses. All students 
within the two groups answered the three sets of questions required to measure 
satisfaction, retention and GPA. The response rate was 100% for the satisfaction and 
retention measurements. Only two students did not answer the GPA question (98.7% 
response rate). 

Students' GPAs were also tabulated for the two student groups. Analysis of 
Variance (ANOVA)and t-test for differences in the means were used as appropriate to 
determine whether the on-site and the hybrid groups differed significantly in 
satisfaction, retention, and GPA. 

RESULTS 



Table: 1. 

Mean Satisfaction and Retention Scores of Hybrid and On-site student groups: 



Groups 


Satisfaction 


Retention 


GPA 


Hybrid 


3.91 


3.67 


3.74 


On-site 


3.75 


3.79 


3.77 


Column Means 


3.83 


3.73 


3.76 



From Table 1, we find that on an initial visual examination, there appears to be no 
significant difference in satisfaction, retention or GPA between the two groups. The 
detailed statistical analysis to confirm this initial finding follows later in the paper. 

Satisfaction 

A test of hypothesis using a one-way ANOVA was conducted. That is, only the factors 
related to satisfaction were considered. Under this condition, the variation was either 
due to the treatments or it was random. The null hypothesis for comparing the mean 
levels of satisfaction was that the mean values were equal for both modalities. 

Ho: pl= p2 

Ha: The mean satisfaction levels were not the same. 
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ANOVA 



Source of 
Variation 


55 


df 


MS 


F 


P- value 


Group of 
Students 


0.024012367 


1 


0.0406667 


3.60219 


0.174512 


Level of 
Satisfaction 


0.0194 


1 


0.02455 


0.77801 


0.360921 


Error 


0.01234333 


1 


0.00 






Total 


0.0392 


3 









Retention 

A test of hypothesis was conducted to determine whether the mean of the two 
student groups differ to a statistically significant degree. 

Table 8 shows the scores of the means of the three levels of retention by the two 
students groups. The null and alternate hypotheses were stated: 

1. Ho: The mean retention levels were the same (pi = p2 = p3). 

Ha: The mean retention levels were not the same. 

2. Ho: The mean of retention levels of the Hybrid and on-site student groups 
were the same (pi = p2). 

Ha: The mean of retention levels the Hybrid and on-site student groups were 
not the same. 



Table: 2. 

Levels of retention scores by the two groups of students/ 





Level One — 
Three Classes 
Completed 


Level Two — 
Ten Classes 
Completed 


Level Three — 
Completing 
Final Course 


Hybrid 


3.78 


3.74 


3.54 


On-site 


3J7 


3.90 


3.77 



A two- factor ANOVA was calculated using a .05 significance level (Table 3). The null 
hypotheses of the mean values of the three retention levels and the two student 
groups was accepted. 



Table: 3 

Two-Way ANOVA results between the two groups of students 
and the three levels of retention: 



ANOVA: Two- 
Factor 
Without 
Replication 
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SUMMARY 


Count 


Sum 


Average 


Variance 


Hybrid 


3 


11.06 


3.68666667 


0.01653333 


On-site 


3 


11.44 


3.81333333 


0.00 


Level One 
Level 


2 


7.55 


3.775 


5E-05 


Two 


2 


7.64 


3.82 


0.0128 


Level 

Three 


2 


7.31 


3.655 


0.02645 


ANOVA 



Source of 
Variation 


55 


df 


MS 


F 


P-vaiue 


Fcrit 


Group of 
Students 


0.0240666 


1 


0.02406667 


3.15973742 


0.0217451 


18.51282 


Levels of 
Retention 


0.0291 


2 


0.01455 


1.91028446 


0.0343609 


19 


Error 


0.0152333 


2 


0.00 








Total 


0.0684 


5 











It can be concluded, based on the sample results that there is no difference in the 
three levels of retention and the two groups of students. The F values for both 
variables were greater than the critical F values. The p-value for the null hypothesis 
regarding the groups of students was 0.217 and .343 for the levels of retention. 
These p-values confirm the hypothesis that the null hypotheses for the two groups of 
students and the three levels of retention should be retained. 

GPA 

The GPA scores are shown in Table 10 for the hybrid and on-site groups of students. 
Results indicate that 96% of the student sample had a GPA of 3.0 or above for both 
groups of students. This high score further supports the initial conclusion that there 
is no significant difference in the "GPA level" between the Hybrid and the on-site 
groups of students. The null hypothesis was that there was no statistically significant 
difference between the means of the groups. 

Ho: The mean GPA levels were the same (pi = p2). 

Ha: The mean retention levels were not the same. 

Results of a t-test to compare the mean GPA values fell within the 95% 
confidence level to support the null hypothesis. The t-score value was 0.452 therefore 
the null hypothesis could not be rejected. 

Table: 4 

GPA scores of Hybrid and On-site group of students. 





N 


Average 

GPA 


Hybrid 


87 


3.74 


On 

Site 


75 


3.77 



CONCLUSIONS 
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The results of this study suggest that students who choose to enroll in courses in an 
on-site format have the same rates of satisfaction and enrollment retention as do 
students that enroll in hybrid courses. This finding is consistent with that of earlier 
studies that reported different types of learning styles and characteristics of 
successful hybrid and on-site learners, and suggests that students enrolled in these 
courses by choice probably possess the attributes likely to make learning a 
satisfactory and constructive experience. Tabulation of the average GPA score 
achieved indicates that 90% of the sample population has a GPA of B- or higher. 
Given this uniformity of scores, it is safe to conclude that, to the degree that these 
are measured by grades, there seems to be no measurable difference between on-site 
and hybrid students GPA results. The individual student's choice of instruction mode 
made no difference in grade outcome. Students enrolled in both forms of instruction 
seem to do well academically and a decision with regard to persisting on their course 
of studies is not influenced by their GPA. 

EXTENDED RESEARCH 

Additional studies could be done focusing on the question of adult students' 
motivation to achieve high scores as well as professors' grading methods. Are adult 
students more motivated? Are professors fair or are they too lenient in their grading? 
In addition, further research on these issues might expand the analysis to other 
campuses within the university and other universities, employ narrower or broader 
measures of student learning, and perhaps focus on specific characteristics of 
different groups as mentioned above, namely nursing, MBA and undergraduate 
students. This could possibly contribute to the observed similarities in behavior 
between the surveyed groups and result. 
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